docs: plan the v1 redesign (architecture + 10 ADRs) - #13

Design-only. No code changes — misc/docs/ only, so the published package is unaffected.

Plans the v1 redesign of s3dol into a dol-based adapter for S3 and S3-compatible object storage, with AWS as the reference semantics. Adds misc/docs/architecture.md and ten ADRs.

Tracking issue: #11 · Deferred scope: #12 · Live bug found on the way: #10

Shape

Three layers, matching azuredol so one adapter in the family reads like the next: connection (credential + endpoint SSOT — lazy, picklable, redacting) → base (Collection→Reader→Store triads, owns the prefix, one error seam, ObjectHandle) → recipes (factories + codec stacks, by composition only). Providers are rows in a preset registry, never subclasses.

Scope is core + large-object I/O; versions/tags/bucket-config/in-flight-uploads as Mappings, async, fsspec and obstore are deferred with a written rule for the s3dol/botodol line.

What the research turned up

Verified against the current release, not inferred:

• explicit endpoint_url is silently dropped when env credentials exist, and explicit credentials are overridden by env — so a store configured for a non-AWS endpoint talks to AWS
• list(store) returns [] on any error — wrong bucket, expired token, missing permission
• writing to a missing bucket creates it, even with make_bucket=False
• url_for presigns with SigV2 (url_for produces SigV2 presigned URLs, which modern S3 and every major S3-compatible provider reject #10) — rejected by modern S3 and every major alternative
• stores are unpicklable; anonymous access to public buckets is impossible; del buckets[name] cascades unpaginated

Two findings that changed the design mid-flight

An adversarial review (three independent lenses + an adjudicating synthesis, all claims executed against real dol/botocore/moto) broke the first draft's central bet, and both corrections are worth calling out because they are the kind of thing that ships silently:

1. Prefix scoping cannot live in a dol wrapper above an absolute-keyed leaf. dol delegates unknown attributes with the outer, unmapped key, so url_for, sub, handle, info, prefixes and delete_many all address the wrong object — delete_many destroys it. Prefix pushdown to ListObjectsV2 is also unimplementable across that seam (measured: 22 LIST requests where a leaf-owned prefix costs 1), and filt_iter resurrects the __len__ that was deliberately omitted.

The first draft cited azuredol as precedent. azuredol's architecture doc describes a design its own code doesn't implement — its base.py does prefix arithmetic in the leaf and uses mk_relative_path_store zero times, which is exactly why it has none of these bugs. ADR-0001 now records the general lesson: a sibling package's design doc is a claim; its source is the evidence.

2. EncodingType='url', which the first draft mandated, causes the corruption it was meant to prevent. botocore already sets it and decodes the response — but the decode is gated on a flag it sets only when the caller didn't pass the parameter. Passing it explicitly drops 5 of 7 test keys out of round-trip. Decision deleted.

Also corrected after review: the value law (normalize is one-shot, not total, for stream refs — so the str rejection stands on decidability alone), singledispatch registration (io.IOBase, never typing.BinaryIO, which silently never fires), the error-classification key ((operation, code, status) — HEAD has no body, so missing-key and missing-bucket are indistinguishable on real AWS and moto masks it), 403-means-absent under the standard least-privilege IAM policy, setdefault silently overwriting an archived object, the import budget, a one-character naming collision, and the conformance suite.

Upstreams

Two dol bugs found here, filed and blocking the implementation (not this PR): i2mint/dol#82 (prefix relativization corrupts non-matching keys — a boundary violation wherever a prefix is a tenant boundary) and i2mint/dol#83 (capability methods delegated with the unmapped key). Both affect every *dol adapter, not just this one.

Compatibility

s3dol.store.S3Store keeps its signature and becomes a deprecated shim removed in v2 — and doubles as the fix-delivery mechanism, since dependents get corrected endpoint/credential resolution without changing a line. Release ordering is specified in ADR-0007 §5, including shipping s3dol.diagnose() first in a 0.1.x patch so a dependent whose data target moves can bisect it.